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One of the issues that have continued to attract the attention of Open and Distance 
Learning (ODL) educators, scholars and researchers at the Zimbabwe Open University 
(ZOU) is the question of quality in the assessment of students' research work. 

This study was part of a series of studies, into issues of quality, currently being conducted 
at the ZOU by the present authors. The present study aimed at developing a systematic 
approach towards quality assessment of students' research work. A content analysis of 
400 research projects submitted by ZOU undergraduate students between 2000-2009 was 
undertaken applying both conceptual and relational analysis. Conceptual analysis was 
based on a checklist of six concepts: research questions, research paradigm, research 
design, data collection instruments, data form, procedures for data analysis and 
presentation and sought to establish the status of students' research work, its gaps and 
areas of saturation. Relational analysis was conducted through the use of a model that 
sought to assess the methodological appropriateness and methodological quality of the 
research projects. 

Major findings of the study revealed a clear distinction between research methodologies 
applied by students in the faculties of Arts & Education, Commerce & Law and Social 
Sciences and those applied by students in the faculty of Physical Sciences. Most research 
projects in the faculty of Physical Sciences, in the department of Agriculture, were 
quantitative by nature, guided by relational research questions and hypotheses. These 
studies used simple experiments to collect numerical data through the use of tests and 
observations, applied statistical analysis of data and presented data through the use of 
statistical tables. Almost all studies in the same faculty, in the department of Geography, 
were case studies and employed mixed methods. Most research studies from the faculties 
of Arts and Education, Commerce and Law and Social Sciences were qualitative, employed 
the descriptive survey research design, used questionnaires and interviews and used both 
quantitative and qualitative techniques to analyse, and present research findings. 
However, a significant number of projects across faculties did not identify the research 
paradigm. 

The present study established a growing trend, amongst qualitative researchers, towards 
the use of mixed methods. Relational analysis revealed that students' projects 
demonstrated a high degree of methodological appropriateness and methodological 
quality. 62 



Appropriate linkages were established between research questions, data collected and the 
methods used in data collection, analysis and presentation. The use of mixed methods also 
enhanced the quality and appropriateness of the research methodology. However, The 
confinement of students' projects to descriptive surveys and to simple experiments 
reflected limited knowledge and low- level research skills in both research students and 
perhaps their supervisors. 

The relational analysis model employed in the present study was found to be very useful in 
the quality assessment of students' research projects. Research students and their 
supervisors can use this model to assess quality of their own research work. The study 
recommends that developers of teaching and learning materials should develop courses 
and materials that adequately equip students with the right knowledge and skills required 
for research. 

There is also need to evaluate the current courses that prepare students for research 
work. These courses are: 

> Introduction To Research Methods 

> Statistics For Educational Research And 

> Computer Application In Educational Research 

INTRODUCTION 

The major concern about Distance Education continues to be its quality (Dellana, Collins, 
and West, 2000). Most research studies focusing on quality have concentrated on 
comparing distance education with conventional education. These studies have found no 
significant difference between distance education and conventional education in terms of 
quality (Phipps and Merisotis, 1999; DeSantis, 2002; Russell, 2000). However, an issue 
that has continued to attract attention of open and distance learning (ODL) educators, 
scholars and researchers is how ODL institutions, whatever their structure, context or 
circumstances, can assess their own quality (Myrdal, 1994). According to Myrdal (1994), 
some of the critical questions that continue to fuel the "distance education" quality debate 
are; What makes distance education an experience that would be described as one of 
quality? How can distance education improve the quality of the ODL it offers? How can an 
institution providing ODL assess its own quality effectively? These are large and difficult 
questions. 

Agreeably, all institutions providing ODL will have some existing systems and procedures 
for assessing the quality of what they do. But not all have addressed the assessment of 
quality within their organizations in a systematic way as much as they need to (Mertens, 
2005). Challenges facing ODL institutions are that procedures for assessing quality can be 
ad hoc, piecemeal, unsystematic, too reliant on individual discretion, and standards of 
practice can be unnecessarily inconsistent and variable. These are some of the challenges 
experienced by lecturers at the Zimbabwe Open University particularly in the assessment 
of students' research projects. Inspired by the need to have a systematic approach that 
could be used in assessing the quality of students' research projects, a team of four 
lecturers at the ZOU conducted a content analysis of 400 research projects submitted by 
undergraduate students between the years2000-2009. 

The study led to the development of a content analysis model the four researchers found 
to be very useful to both research students and their supervisors in the quality assessment 
of students' research projects. 
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This article discusses how this model was developed and its application in the assessment 
of students' research projects. 

BACKGROUND OF THE STUDY 

The Zimbabwe Open University (ZOU) is an Open and Distance Learning (ODL) institution 
in Zimbabwe, established to cater for a substantial component of people who, by design or 
unintentionally, could not be accommodated in conventional universities, by offering them 
the opportunity to study in their homes and in their workplaces through distance 
education. The ZOU was established on 1 st March 1999 through an Act of parliament 
(Chapter 25:20), with an initial enrolment of 624 students registered for the Bachelor of 
Education degree programme. By 2004 ZOU had become the largest university in the 
country and second largest in Southern Africa compared to University of South Africa 
(UN1SA), with a student enrolment of approximately 13 000. However, the student 
population, in the year, 2010, dropped to approximately 10 000. During the time of this 
study, in 2010, ZOU had four faculties; 

> the faculty of Arts and Education, 

> the faculty of physical Sciences, 

> the faculty of Commerce and Law and 

> the faculty of Social Sciences, offering 30 undergraduate degree programmes, 3 
diploma courses, 5 masters degree programmes and 2 doctoral degrees. 

The ZOU teaches its students using multi media methods that include the module (print 
text), face-to-face tutorials and ICTs. Students are assessed through the use of 
assignments, tests, examinations and research projects. 

The present study is part of a longitudinal study to investigate issues of quality in the 
assessment of students through the research projects. Researchers at the ZOU have 
identified a number of challenges facing research students and their supervisors 
(Nyawaranda (2005; Kangai and Mapolisa (2008). 

According to Nyawaranda, (2005), challenges include the following: 

> Students failing to complete research projects in time 

> Low submission rate of 40% 

> Students working on non-researchable topics 

> Plagiarism 

> Poor quality of research projects 

> Lack of competence (by supervisors) in approving students' research topics 

> Inconsistence in marking students' research projects 

> Lack of uniformity in the supervision and marking of students' research projects 

These challenges have impacted negatively on the quality assessment of students' 
research projects. It is against this background that the present content analysis study 
was undertaken. 

Purpose of the Study 

The purpose of the study was to conduct a content analysis of research projects submitted 
by ZOU undergraduate research students in order to establish their present status, gaps, 
and areas of saturation and to develop a model that could be used in the quality 
assessment of these projects. 
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Statement of the Problem 

One of the topical issues that have provoked heated academic debate in programme 
management meetings at the ZOU is the question of quality in the assessment of students' 
research projects (Izuagie, 2001). 

The central issue confronting research students and their supervisors at the ZOU is how to 
achieve quality, effectiveness and productivity in the new changing environment. The 
research problem is that not all projects' supervisors and markers have addressed the 
assessment of students' projects in a systematic way. For example double marking of 
projects have produced extreme differences in marks awarded to the same project. The 
present study sought to assess, through a content analysis, the quality of research 
projects submitted by ZOU undergraduate students and to develop a content analysis 
model that could be used for quality assessment of these projects. 

Research Questions 

The present study was guided by the following research questions: 

1. Do ZOU undergraduate students have a bias in the choice of research topics, 
research paradigm, and research design and data collection methods? 

2. What is the status (methodological appropriateness and methodological quality) of 
students' research projects? 

Importance of the Study 

Research is the process by which new knowledge is created. As such it influences the 
direction in which an academic field or a community moves. The findings of the current 
study are of interest to many groups in distance education; however, we wished to provide 
this information primarily for the benefit of research students and their supervisors who 
should be aware of how quality can be enhanced during supervision and assessment of 
research projects. The information presented in this article is also of great benefit to 
lecturers who are involved in the teaching of research theory and methodology. The 
findings are also of great value to developers of distance education materials. 

LITERATURE REVIEW 

The research Project: Definition and structure 

The present study is a content analysis of research projects submitted by ZOU 
undergraduate students (2000 - 2009). Reckase (1995) defined a research project as a 
purposeful collection of student work that exhibits to the student and others, the student's 
efforts, progress, or achievement in a given area. The major goal of project work is to 
assess the ability to apply knowledge to solve real-life problems. The student must take 
considerable control over the assessment through planning and applying knowledge in 
perhaps new and different ways. Projects, thus attempt not only to assess the end 
products, but to some extent, the process that went into creating them as well. 

THEORETICAL FRAMEWORK 

Although there is no universally accepted format for the research process most studies 
adhere to the sequence of scientific enquiry. Scientific enquiry is the search for knowledge 
by using recognized methods in data collection, analysis and interpretation. The scientific 
method is usually a sequential research process. The typical steps in the scientific method 
are: 
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> Define a problem 

> state the hypotheses to be tested 

> collect and analyse data 

> interpret the results and draw conclusions about the problem (McMillan and 
Schumacher, 1993). 

Research has found that different universities and even faculties in the same university 
may use the same process with minor modifications. Students' research projects at the 
ZOU are structured following seven stages; 

> The research topic/title 

> Research questions/Hypotheses 

> Literature review 

> Research methodology 

> Data collection 

> Data analysis, presentation and discussion of findings 

> References 

The Research Topic/Title 

Most ZOU degree programmes require students to undertake research and submit a 
research project report within a period of 12 months. The research projects are 
unstructured. This means students choose their own topics, research paradigm, research 
design and data collection instruments with minimum guidance from their supervisors. 
The research process begins with the student identifying a research problem and 
formulating a research topic which he or she then submits to the programme coordinator 
for approval. The role of the coordinator is to assess whether the topic is researchable, 
relevant to the programme for which it is undertaken, feasible, concise, and unambiguous, 
of public interest and clearly delimits the area of study. The coordinator then allocates the 
student to a supervisor on the basis of the supervisor's proximity and expertise in the 
student's research topic. Bogdan and Biklen (1992) has noted that right at the initial 
stage, quite often, students fail to define a research problem from either a conventional or 
technical meaning. Kratwohl (1985) identifies lack of training and experience in research 
methods as an obstacle to students' progress in research. Thus the quality of a research 
project can be affected by the choice of a research topic, lack of training and the use of 
inappropriate methods. The present study sought to establish the methodological 
appropriateness and methodological quality of projects submitted by ZOU undergraduate 
students from 2000-2009. 

Research Questions/Hypotheses 

Research studies may state the research problem in question form or as a hypothesis. 
According to Hedrick,et. al., (1993) the research topic operationalises objectives of the 
proposed research which then lead to research questions or hypotheses. Research 
questions clarify and focus the researcher towards what information (data) needs to be 
collected, from what sources and under what conditions. The researcher's worldview 
(Paradigm) influences the nature of the research questions and proposed methodology 
(Mertens, 2005). The question format is often preferred because it is simple and direct. 
Psychologically, it orients the researcher to the immediate task: to develop a design to 
answer the question (Mertens (2005). A research hypothesis is a tentative statement of 
the expected relationship between two or more variables. A hypothesis directs the study. 
Actually the study is undertaken in order to either accept or reject the hypothesis. 
Research questions are mainly associated with qualitative research and hypotheses are 
associated with quantitative studies. 
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The present study sought to establish whether students' research projects were guided by 
research questions or hypotheses. Borg and Gail (1989) put research questions into two 
categories: descriptive questions and relational questions. The former are based on the 

question "What is.?, while the relational studies are predicted by questions such as 

"What factors determine, cause or lead to....?". This current study is based on Hedrick et. 
al., (1993) taxonomy of research questions. Hedrick et. al., (1993) posit that research 
questions can be classified into four types: descriptive, normative, co relational and 
impact. 

Descriptive research questions are designed to produce information about what is or has 
been happening in relation to the target of the research. Normative research questions go 
beyond description and require that information generated in response to descriptive 
research questions be compared with some standard or expected observation. Correlative 
research questions are used to identify relationships to enable the explanation of 
phenomenon. Hedrick, et. al., (1993) point out that data derived in response to such 
questions indicates the strengths and direction of a relationship between two or more 
variables, not causality. Impact research questions aim at identifying effects, to establish 
causal links between an independent variable (the intervention) and a dependent variable 
(the anticipated change). 

Thus the type of research question determines the type of data to be collected for a 
particular study. Research data can be either in numbers or words. According to Leedy 
(1993) the nature of the data and the research question dictate the methodology that 
must be employed in a particular study. 

Research that collects, analyze and present data in the form of numbers is said to be 
quantitative. Research that uses words is qualitative. Research that collects both 
quantitative and qualitative data is said to use mixed methods. 

Research paradigms 

A paradigm is a way of looking at the world. It is composed of certain philosophical 
assumptions that guide and direct thinking and action of the researcher. The labels 
quantitative, qualitative and mixed methods are linked to particular research paradigms. 
Lather (1992) and Lincoln and Guba (2000), have identified four major paradigms that 
have influenced educational research: Postpositivism, Constructivist, Transformative and 
Pragmatic. The present study sought to establish the research paradigm most preferred by 
undergraduate students and the extent to which the paradigms were related to the other 
concepts in the research projects (research design, data collection instruments, form of 
data collected and procedures for data analysis and presentation. 

Postpositivism 

This was the dominant paradigm that guided early educational research. It is commonly 
referred to as the quantitative paradigm. Positivism is based on the rationalistic, empiricist 
philosophy that originated with Aristotle, Francis Bacon, John Lock, August Compte, and 
Emmanuel Kant (Mertens, 2005). Positivism assumes that the social world can be studied 
in the same way as the natural world and there is a method for studying the social world 
that is value free and that explanations of a causal nature can be provided and events or 
observations are influenced by a single theory or paradigm. 

Constructivist 

The proponents of constructivist paradigm were Campbell and Stanley (1963, 1966), Cook 
and Campbell (1979). 
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According to Cook and Campbell (1979), constructivist paradigm rejects the position that 
events or observations are influenced by a single theory or paradigm. The basic 
assumptions guiding the constructivist paradigm are that knowledge is socially 
constructed by people active in the research process and that researchers should attempt 
to understand the complex world of lived experiences from the point of view of those who 
live it (Schwandt 2000). The constructivist paradigm, also known as qualitative paradigm, 
emphasizes that research is the product of the researchers and cannot be independent of 
them (Mertens, 2005). 


Transformative 

The transformative paradigm assumes that there are multiple realities shaped by social 
interaction. There is an interactive link between researcher and participants. Knowledge 
is socially and historically situated. Research studies adopting the transformative 
paradigm include action research, critical theory, participatory research and feminist 
theory. Researchers following the transformative paradigm fix themselves self¬ 
consciously as participatory activists and believe that when one immerses oneself in the 
activity, there is likelihood that one elicits more data (Lincoln & Guba, 2000). The main 
focus of transformative researchers is to seek to unearth, disrupt, and transform existing 
ideological and or institutional arrangements. This is the research paradigm usually 
adopted by political activists and gender lobbyists when they seek to change gender and 
political ideologies. 

Pragmatism 

Pragmatism is a paradigm that provides the underlying philosophical framework for mixed 
methods research (Maxcy, 2003; Patton, 2002; Tashakkori and Teddlie, 2003). These 
philosophers rejected the scientific notion that social science enquiry was able to access 
the truth about the real world solely by virtue of a single scientific method. To this end, 
the most appropriate paradigm is the pragmatic paradigm in which the researcher is not 
confined to one research paradigm (Patton 2002). When analyzing methodological 
appropriateness and methodological quality of research projects one must be aware of the 
paradigmatic debates that currently exist between quantitative and qualitative 
researchers. 

The Quantitative and Qualitative Paradigm Debate 

Most research work is guided by either the quantitative (logical positivism) or qualitative 
(constructivist) research paradigm or a mixture of the two (pragmatism) (Guba, 1978). 
Researchers have long debated the relative value of qualitative and quantitative inquiry 
(Patton, 1990). Each approach represents a fundamentally different inquiry, and 
researcher actions are based on the underlying assumptions of each paradigm. 

Quantitative research reflects the traditional scientific approach to problem solving. It 
uses experimental methods and numerical data to test hypothetical generalizations. It 
assumes that there is a single reality that can be broken down into variables. The purpose 
of this type of research is to test hypotheses that have been developed before the research 
project started and to form conclusions that can be generalized to other situations. The 
emphasis in this approach is upon measurement, comparison, and objectivity. Qualitative 
research, broadly defined, means any kind of research that produces findings not arrived 
at by means of statistical procedures or other means of quantification. Qualitative 
research uses a naturalistic approach that seeks to understand phenomena in context- 
specific settings. 
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Where quantitative researchers seek causal determination, prediction, and generalization 
of findings, qualitative researchers seek instead illumination, understanding, and 
extrapolation to similar situations. The most obvious distinction between quantitative and 
qualitative research is the form of data collection and presentation. 

The quantitative approach has dominated the field of education since the 1950's and is 
considered by some as the major approach to educational research. Proponents of the 
quantitative paradigm include Dodds, Lawrence, & Guiton, 1984); Kember & Dekers, 
1987); Osman & Wagner, 1987). These researchers maintain that quantitative methods 
are inherently preferable to non-quantitative methods. Contrary to this view. Smith and 
Manning (1982) suggest that qualitative research is better for initial exploratory research 
into unknown or unfamiliar phenomena. 

It enables the researcher to develop concepts, conjectures, interpretations and theories 
empirically grounded in the investigated systems. In the real world, events cannot be 
teased out from the context in which they are inextricably embedded, and understanding 
involves the relationships among all of the many parts of the whole. 

Thus qualitative methods- which emphasize both inner and outer knowledge of man in his 
world- are preferable (Filstead, 1970). This argument has led to the present confusion 
between the quantitative and qualitative paradigms (Owens, 1982). The prevailing 
theoretical understanding is that a combination of the two paradigms can generate more 
powerful analyses than either approach could generate alone (Miles, 1983). Researchers 
such as Strauss & Corbin (1990); Patton, (1990), believe that qualitative and quantitative 
research methods can be effectively combined in the same research project. 

In support of this view, Russek and Weinberg (1993) claim that by using both quantitative 
and qualitative data, their study of technology-based materials for the elementary 
classroom gave insights that neither type of analysis could provide alone. However, the 
aim of the present study was not to enter into the 'paradigm war' but to contribute 
knowledge towards quality assessment of students' research work. This article in no way 
suggests that one paradigm is inherently superior to the other but that both have an 
appropriate context. 

A research paradigm (quantitative, qualitative or mixed methods) has a direct influence in 
the choice of research design to be adopted for a particular study. Methodological 
appropriateness is established when there is a perfect link between the research paradigm 
and the research design. 

Research Design 

According to McMillan and Schumacher (1993), research design refers to the plan and 
structure of the investigation used to obtain evidence to answer research questions. The 
design describes the procedures for conducting the study, including when, for whom, and 
what type of data will be obtained. In actual fact, it is the nature of data to be collected 
that dictates the research design that must be employed in a particular study. The design 
indicates how the research is set up; what happens to the subjects and what methods of 
data collection are used. 

The purpose of the design is to provide the most valid, accurate answers possible to the 
research question. Since there are many types of research questions and many types of 
designs, it is important to match the design with the research question. For the purposes 
of the present study, research designs were classified according to four categories; true 
experimental, quasi-experimental, non-experimental and mixed methods. 
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The current study sought to establish the most preferred research design by ZOU students 
and the appropriateness of the designs used. 

Data Collection Techniques 

Another way of classifying research besides research designs eg,.experimental, non- 
experimental, is to examine the technique used in the study to collect the data. There are 
basically six ways to collect data: observations questionnaires, interviews, documents, 
tests, and unobtrusive measures (Mcmillan and Schumachar, 1993). 

These techniques can be classified as either quantitative or qualitative. Quantitative 
techniques use numbers to describe phenomenon while qualitative techniques use 
narrative descriptions. Most of the techniques can be used with any of the research 
designs. Quantitative techniques are used with experimental, descriptive and correlational 
designs (Mcmillan and Schumachar, 1993). 

Examples of quantitative techniques include structured observations, standardised 
interviews, tests, questionnaires, unobtrusive measures. 

Qualitative techniques are used with descriptive surveys, ethnographic, historical, content 
analysis and case study designs. Examples of qualitative techniques include ethnographic 
observations, ethnographic interviews, and documents. The present study sought to 
establish whether the data collection techniques used in the projects were appropriate to 
the research designs. 

METHODOLOGY 

The present study employed both quantitative and qualitative research techniques and 
was guided by grounded theory. Grounded theory relates abstract concepts to propose a 
theory as an explanation of the phenomena. 

The theory is "grounded" in that it is developed from the data in contrast to testing a 
theory from the literature. Data are presented to describe each concept and its 
relationships to the other concepts. 

Research Design 

The present study involved content analysis of research projects submitted by ZOU 
undergraduate students. There are two general categories of content analysis: conceptual 
analysis and relational analysis. Both types of content analysis were applied. Conceptual 
analysis focused on establishing the existence and frequency of certain concepts in the 
projects. 

The concepts were chosen for examination and the number of their occurrences (quantity) 
within the projects recorded. In this study conceptual analysis was undertaken by using 
open coding. Open coding is the part of analysis that pertains specifically to naming and 
categorizing phenomena through close examination of data (Mertens, 2005:424). In order 
to come up with a checklist of concepts and codes, we initially analyzed 50 research 
projects. 

Our analysis produced a checklist of six subcomponents of the research project: research 
questions, research paradigm, research design, data collection instruments, data form, 
procedures for data analysis and presentation. These concepts were categorized and 
coded as presented in Table: 1. 
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Table: IChecklist of Concepts, Categories and Codes 


Concept 

Category 

Code 

Research Questions 

1. Descriptive 

D 


2. Normative 

N 


3.Correlational 

C 


4.1mpact 

I 

Data Form 

1.Numbers 

N 


2. Words 

W 

Research Paradigm 

l.Post Positivist (Quantitative) 

PP 


2.Constructivist (Qualitative) 

C 


3.Transformative 

TP 


4.Pragmatic (Mixed Methods) 

MMP 

Research Design 



1. True Experimental 

• Post-test equivalent group 

Pt 

(Quantitative) 

• Pretest-posttest 

PtPt 


• Solomon four 

SF 


• Analytical survey 

AS 

3. Quasi-Experimental 

- 


(Quantitative) 


Q 

4. Non-Experimental 

• Descriptive survey 


(Qualitative) 

• Ex post facto 



• Case study 

DS 


• Content analysis 

EPF 

5. Mixed Methods 

• Historical 

CS 

(Quantitative- (Qualitative) 

- 

CA 

H 



MMD 

Data collection instruments 

• Questionnaires 

Q 


• Interviews 

i 


• Observations 

0 


• Tests 

T 


• Documents & records 

D&R 


Reliability of the Methods 

We checked the reliability of the coding a (95% agreement is suggested; .8 for Cohen's 
kappa). 

To establish the dependability of the categorization decisions, one researcher coded all the 
items and a second researcher independently coded a 10% random sample of those items. 
The proportion of simple agreement between the coders was 0.96. 

Sample 

Data for the current study was collected from a sample of 400 research projects, which 
were submitted by ZOU undergraduate students between te years 2000-2009. The 
sampling techniques involved the following steps. 

First, we identified the four faculties in the university: Arts & Education, Social Sciences, 
Commerce & Law and Physical Sciences. From each faculty two undergraduate degree 
programmes were randomly selected using the hat method. Each Faculty has a minimum 
of four undergraduate degree programmes. From each degree programme, 50 research 
projects were randomly selected. (See Table: 2) 
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Table: 2 

Random sample of research projects content 
analyzed for the current study 


FACULTY 

DEGREE PROGRAMME 

PROJECTS 

Arts & Education 

1. Bachelor of Educational Management 

50 


2. Bachelor of Arts, English & Communication 

50 

Social Sciences 

1. Bachelor Science (Honours) in Counselling 

2. Bachelor of Science (Honours) in Special 

50 


Education 

50 

Commerce & 

1.Bachelor of Commerce in Human Resource 

50 

Law 

Management 



2. Bachelor of Commerce in Marketing 

50 

Physical Sciences 

1. Bachelor of Science Degree in Agriculture 

2. Bachelor of Science Degree in Geography 8i 

50 


Environmental Sciences 

50 


TOTAL 

400 


Using the checklist of concepts and codes, the four researchers conducted a conceptual 
and relational analysis of a sample of the 400 research projects submitted by ZOU 
undergraduate students from 2000-2009. The first objective of the present study was to 
establish the status of students' research work, its existing gaps and areas of saturation.In 
order to achieve this objective, we analyzed each project and identified the existence of 
the concepts and their frequency. 

The analysis focused on the following concepts: research questions, research paradigms, 
research designs, and data collection methods. 

The second objective of the study was to undertake a relational analysis of the research 
projects in order to establish the quality and appropriateness of the methods applied. 

Findings of the present study are discussed below with reference to the research questions 
that guided the study, the prevailing research theory and findings of previous research 
studies. 

DISCUSSION OF FINDINGS 

Dominant Research Questions and Hypotheses 

The first concept we analysed was the research question or hypotheses that guided each 
study. 

Findings of the current study revealed that, across faculties, the majority of research 
projects one hundred and ninety-five (48.75%) used descriptive research questions, 
followed by ninety-four (23.5%) impact research questions. Sixty-five (16.25%) 
correlational questions and forty-six (11.5%) used normative research questions (See 
Table. 3 below). 

Most studies in the department of Agriculture were guided by hypotheses and studies in 
the department of geography used both hypotheses and research questions. 77 










Table: 3 

Research questions most preferred by ZOU undergraduate students 


FACULTIES 

RESEARCH QUESTIONS 

Descriptive 

Normative 

Correlation 

Impact 

N 

% 

N 

% 

N 

% 

N 

% 

Arts & Education 

53 

13.25 

8 

2 

25 

6.25 

14 

3.5 

Social Sciences 

80 

20 

14 

3.5 

0 

0 

6 

1.5 

Commerce & Law 

46 

11.5 

11 

2.75 

14 

3.5 

29 

7.25 

Physical Sciences 

16 

4 

13 

3.25 

26 

6.25 

45 

11.25 

Totals 

195 

48.75 

46 

11.5 

65 

16.25 

94 

23.5 


Studies in the faculties of Arts & Education, Commerce & Law and Social Sciences were 
mainly descriptive hence they were guided by descriptive research questions. Descriptive 
studies are mainly used to describe peoples' characteristics such as their attitudes, beliefs, 
habits, values, demographics, behaviours, opinions, desires, ideas, and other types of 
information. 

Our findings agree with Mcmillan & Schumacher (1993) who have noted that descriptive 
research is popular in social research because it does not involve manipulation of 
independent variables. 

Descriptive research questions typically ask "what is "and imply a survey research design 
(Mcmillan & Schumacher, 1993:86). Mertens, (2005:107) says descriptive researches are 
designed to produce information about what is or has been happening in relation to the 
target of the research. Borg and Gall (1996 p. 373) characterize descriptive research as 
"the most basic of methods, which involves simply describing the characteristics of 
individuals or other phenomena." 

The use of descriptive research questions by ZOU students, therefore, may be an indicator 
or a measure of students' level of research skills and the quality of training and project 
supervision availed to the students. Impact research questions were second most 
preferred and were popular amongst research students from the faculty of Physical 
Sciences. Research projects from Agriculture were mainly experimental, hence they used 
impact questions and hypotheses (11.25%) and correlational questions (6.25%). 

The Dominant Research Paradigm 

The second concept we analysed was the research paradigm. The present study revealed 
that quantitative research dominated in the physical sciences, particularly in Agriculture 
(72%), and the qualitative paradigm influenced a significant number of studies in the 
faculties of Arts & Education (B.Ed, 10% and English, 18%), Commerce & Law (Human 
Resource, 22% and Marketing, 16%), and Social Sciences (Counseling, 20% and Special 
Education, 12%). 

Another finding was that there was a growing trend towards the use of mixed methods 
(B.Ed, 54%; English, 46%; Human Resources, 52%; Marketing, 40%; Counselling, 44% 
and Special Education, 58%). 

The pragmatic paradigm was more dominant in the department of Geography with sixty- 
six percent of the projects applying both quantitative and qualitative techniques (See 
table 4 below). 
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Table: 4 

The Dominant Research Paradigm 


Paradigms 

Faculties 

Arts& Educ. 

Com 8i Law 

Social Science 

Physical Science 

B.Ed 

Eng. 

HR | Market 

Coun | Sped 

Agric | Geo 

N 

% 

N 

°/o 

N 

% 

N 

% 

N 

% 

N 

°/o 

N 

°/o 

N 

% 

Positivism 

(Quantitative) 

3 

6 

4 

8 

2 

4 

9 

18 

8 

16 

6 

12 

36 

72 

1 

2 

Constructivist 

(Qualitative) 

5 

10 

9 

18 

11 

22 

8 

16 

10 

20 

6 

12 

0 

0 

2 

4 

Pragmatism 
(Mixed Method) 

27 

54 

23 

46 

26 

52 

20 

40 

22 

44 

29 

58 

2 

4 

33 

66 

No Paradigm 

15 

30 

14 

28 

11 

22 

13 

26 

10 

20 

9 

18 

12 

24 

4 

8 

Total 

50 

LOO 

50 

100 

50 

50 

50 

100 

50 

100 

50 

100 

50 

100 

50 

100 


Another interesting finding was that 22% of the projects did not identify a paradigm. 
Leaders in the field of research do not all do not agree as to the need to acknowledge an 
underlying paradigm, nor do they agree on the role that such paradigms serve in the 
research process. 

Whilst Patton (2002) says research paradigms are unnecessary and possibly handicapping, 
Schwandt, (2000), argues that research paradigms are inescapable. 

The need for a paradigm is also supported by Mertens (2005), who maintains that a 
researcher's theoretical orientation (paradigm) has implications for every decision made in 
the research process, including choice of methods. Skrtic (1991), argues that the absence 
of a research paradigm, renders the research atheoretical since it rests on an unexamined 
and unrecognized theory. Thus, the fact that 22% of ZOU research students from the 
faculties of Arts & Education, Commerce & Law and Social Sciences proceeded to 
undertake their studies without acknowledging their theoretical paradigm or philosophical 
assumptions (whether by design or otherwise) raises questions pertaining to the quality of 
training and supervision research students receive from their tutors. 

The present study shows that gaps of knowledge and skills with regards to the role of 
research paradigms still exist amongst ZOU research students and their project 
supervisors. The question that comes to mind is "Do ZOU research students fully 
understand the role played by paradigms in research?" We resolved to address this 
question in a follow up study. 

The Dominant Research Design 

The third concept we analysed was the research design. We based our analysis on Borg 
and Gall's classification of research questions. According to Borg and Gall (1989) there are 
two categories of research questions- descriptive and relational. Descriptive questions 
lead to non- experimental designs and relational questions lead to experimental designs 
and the use of both leads to mixed methods. We, therefore, identified and classified 
research designs as experimental, non-experimental or mixed methods. Our findings 
revealed that 72% of research projects from the faculty of Physical Sciences, Agriculture 
in particular, adopted experimental research designs. 
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Research projects from Geography (66%) were mostly case studies and applied mixed 
methods. Research projects from the faculties of Arts & Education, Commerce & Law, 
Social Sciences employed non-experimental research designs confined to descriptive 
surveys (89%), case studies and ex post facto studies. 

This study revealed a clear distinction between research methodologies used in social 
sciences and those used in physical sciences. 

Whilst the descriptive survey research design dominated in almost 89% of studies in the 
faculties of Arts & Education, Commerce, and Social Sciences, experimental designs were 
mostly applied in Agriculture. 

Our findings agree with Mertens (2005); Borg and Gall (1996); and McMillan 8i 
Schumacher (1993) who have noted that descriptive surveys are the most basic of 
methods, popular and are used pervasively in social research. 

Descriptive surveys are mainly used to describe peoples' characteristics such as their 
attitudes, beliefs, habits, values, demographics, behaviors, opinions, desires, ideas, and 
other types of information. 

Certainly, there is a place for descriptive designs in educational research, but whether this 
should occupy almost 90% of students' research at the ZOU is a topic for discussion and 
further research. 

Perraton (1988) argues that the confinement of students' research projects to one or two 
research designs is a function of the lack of theoretical foundations in research 
methodology. The present findings, therefore, fuel academic debate about the level of 
students' research skills and the quality of training and supervision they receive from their 
project supervisors. 

Data Collection Instruments 

The fourth concept we analysed involved data collection instruments. Our analysis 
revealed that the questionnaire was the most preferred data collection instrument in the 
faculties of Arts & Education (35%), Social Sciences (36%), and Commerce 8i Law (35%). 
(See Table: 4). 

A combination of the questionnaire and interviews was also equally preferred. Arts & 
Education (35%), Social Sciences (21%), and Commerce & Law (38%). 

Studies in the faculty of Physical Sciences, particularly in the department of Agriculture 
applied tests and observations (42%), tests and documents (16%) and tests and 
questionnaires (10%). 

This study revealed a clear distinction between data collection instruments applied in social 
research-qualitative studies in (Arts 8i Education, Commerce & Law and Social Sciences) and 
quantitative studies in natural sciences. 

This is because whilst quantitative research emphasizes objective measurements and 
collection of numerical data through tests, observations, questionnaires or surveys 
(Saunders et.al., 1997), qualitative research focuses on understanding social phenomena 
through interviews and personal comments (Strauss & Corbin, 1990). 
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Table: 5 

Data collection instruments mostly preferred 
by ZOU undergraduate research students 


DATA COLLECTION 
TECHNIQUES 

FACULTIES 

Arts & 
Education 

Social 

Sciences 

Commerce & 
Law 

Physical 

Sciences 

N 

% 


% 

N 

% 

N 

% 

Questionnaires only 

35 

35 

36 

36 

35 

35 

5 

5 

Test only 

0 

0 

0 

0 

0 

0 

0 

0 

Interviews only 

13 

13 

23 

23 

15 

15 

15 

15 

Observation only 

0 

0 

2 

2 

0 

0 

2 

2 

Documents 

2 

2 

0 

0 

5 

5 

0 

0 

Group Discussion 

0 

0 

5 

5 

2 

2 

0 

0 

Questionnaire & Interviews 

35 

35 

21 

21 

38 

38 

10 

10 

Questionnaire, Interview & 
Observations 

10 

10 

1 2 

1 2 

4 

4 

0 

0 

Questionnaires & Documents 

5 

5 

1 

1 

1 

1 

0 

0 

Test & Interviews 







1 

1 

Tests & Questionnaire 







10 

10 

Tests & Documents 







16 

16 

Tests & Observations 







42 

42 


Our findings show that most of the qualitative studies in the faculties of Arts & Education, 
Commerce and Law used a combination of the questionnaire and the interview because 
they collected both quantitative and qualitative data. 

RELATIONAL ANALYSIS 

The second objective of the present study was to conduct relational analysis of the 
research projects submitted by ZOU undergraduate students between 2000-2009 in order 
to assess methodological appropriateness and methodological quality. Relational analysis 
of the research projects was conducted through the use of axial coding and selective 
coding (Mertens, 2005: 424). Axial coding is the part of the analytic process in which the 
researcher puts the parts of the data identified and separated in open coding back 
together to make connections between categories (Mertens, 2005: 424). Selective coding 
involves the process of selecting one core category (the story line) and relating the other 
categories to it (Mertens, 2005). Strauss and Corbin, (1990) use the term paradigm to 
refer to the storyline. For the present study we first identified the research question as the 
core category (storyline) and the rest of the concepts as sub-concepts. We then developed 
a model that linked these sub-concepts to the storyline (research question). (See Figure: 1 
below) By using the relational analysis model that we had developed, we classified (for 
each project) the research question as descriptive or relational (differences, impact, co 
relational) and the type of data collected as qualitative, quantitative or both. 

We then classified and analyzed the research paradigm, research design, and data 
collection instruments and data analysis procedures under these three paradigms. To 
establish methodological appropriateness and methodological quality, we analyzed the 
links between the research question (core concept) and the following sub-concepts: data 
collected, research paradigm, research design, data collection instruments and techniques 
employed in the analysis and presentation of research findings. 
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Figure: 1 

Relational Analysis Model for research projects 

Our analysis led to the discovery of four categories into which research studies by ZOU 
undergraduates could be classified: 

> Truly qualitative research, 

> Truly quantitative research, 

> Truly Mixed methods research and 4) Partially Mixed Methods (Table 6) 
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Table: 6 

Types of Research studies at the Zimbabwe Open University 


Paradigm 

Faculties 

Arts & 
Education 

Commerce 

Law 

Social 

Sciences 

Physical 

Sciences 

Total 


N 


N 


N 


N 


N 

% 

Truly Quantitative 

7 


11 


13 


44 


75 

18.75 

Truly Qualitative 

14 


18 


15 


2 


49 

12.25 

Truly Mixed Methods 

3 


1 


2 


34 


40 

10 

Partially Mixed Methods 

47 


46 


51 


4 


148 

37 

No Paradigm 

29 


24 


19 


16 


88 

22 

Total 

100 


100 


100 


100 


400 

100 


Truly qualitative research 

Our study revealed that 49 (12.2 5%) of the studies were purely qualitative. These studies 
came from the faculties of Arts & Education, Commerce & Law and Social Sciences. These 
studies were guided by descriptive research questions, employed descriptive surveys or 
case study research designs, collected data in words, analysed data in context and 
presented findings in words. Only two (0. 5%) research projects from the faculty of 
Physical Sciences were qualitative. Patton (2002), notes that the choice of qualitative 
methods might be appropriate in many educational research studies because they are 
based on humanistic values. 

Truly quantitative research 

Quantitative research accounted for 75 (18.75%) studies of which 36 (9%) came from 
Agriculture and the rest from faculties of Arts & Education, Commerce & Law and Social 
Sciences. Most of the projects from the faculty of Physical Sciences, Agriculture in 
particular were guided by hypotheses or relational (differences, impact, correlational) 
research questions. 

These studies were mainly quantitative, employed experimental research designs, 
collected numerical data through the used tests and observations, analysed data using 
statistical packages such as SPSS and presented research findings using frequency tables, 
statistics, graphs and percentages. 

These projects demonstrated a high degree of methodological appropriateness and 
methodological quality in that quantitative data was collected, analysed and presented 
through the use of quantitative methods. 

Fully integrated mixed methods 

Our analysis revealed that 40 (10%) research projects used mixed methods throughout 
the research process. These studies were mainly from the department of Geography 33 
(8.25%) and the rest from the faculties of Arts & Education, Commerce & Law and Social 
Sciences. 

Almost all studies from Geography, indicated that they were influenced by both 
quantitative and qualitative research paradigms, employed case study research design, 
collected both numerical and textual data, analysed and presented data using both words 
and numbers. Teddlie and Tashakkori (2002) describe the following characteristics as 
those of a truly mixed approach methodology. 78 





















> It would incorporate multiple approaches in all stages of the study (i.e problem 
identification, data collection, data analysis and final inference) 

> It would include a transformation of the data and their analysis through 
another approach eg. content analysis of data followed by a quantitative 
analysis of the same data. 

In this study these characteristics were revealed in the projects from Geography. 

Partial Mixed methods 

Our analysis revealed that 148 (37%) research projects from mainly the faculties of Arts & 
Education, Commerce & Law and Social Sciences employed mixed methods. These studies 
initially indicated that they were qualitative, employed the descriptive survey research 
design but used questionnaires and interviews to collect data in numbers and words. 
Lincoln and Guba (2000) identify qualitative methods as the preferred methods for 
researchers working in the constructivist paradigm. However, they also recognize that 
quantitative methods can be used within this paradigm when it is appropriate to do so. 

According to Mertens (2005) qualitative and quantitative data collection can occur in 
parallel form or sequential form. Mixed methods can also involve the conversion of 
qualitative data to a quantitative form or vice versa. According to Mertens (2005) mixed 
methods can be applied at four different levels: problem identification, data collection, 
data analysis, and discussion of research findings. 

Greene and Caracelli (2002) argue that social researchers mix methods to a varying 
degree at various points in their research and still call their work mixed method research. 
However ZOU undergraduate researchers called their research work qualitative. 

These findings have a bearing on the current 'quantitative - qualitative' paradigmatic 
debate. Proponents of mixed methods (Patton, 1990; Strauss & Corbin 1990) believe that 
qualitative and quantitative research methods can be effectively combined in the same 
research project. 

In fact, Patton (1990) advocates for a "paradigm of choices" that seeks "methodological 
appropriateness as the primary criterion for judging methodological quality. 
Methodological appropriateness, Patton (1990) argues, will allow for a "situational 
responsiveness" that strict adherence to one paradigm or another will not. 

Patton is also supported by Lincloln and Guba (2000) who note that many changes have 
occurred in the status of paradigms and choice of methods over the last decade such that 
various paradigms are beginning to "interbreed". Russek and Weinberg (1993) have also 
argued in support of mixed methods. 

They claim that by using both quantitative and qualitative data, their study of technology- 
based materials for the elementary classroom gave insights that neither type of analysis 
could provide alone. 

However, Mertens (2005) argues that the field of research has not yet reached the point of 
full integration of paradigms. Our findings support Mertens (2005) in that all the studies 
that made use of mixed methods in the faculties of Arts & Education, Social Sciences and 
Commerce and Law) except for Geography, initially indicated that the studies were 
qualitative but went on to apply both quantitative and qualitative techniques and 
procedures during data collection, analysis and presentation. 
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CONCLUSIONS AND DIRECTIONS FOR FUTURE RESEARCH 


This study was a content analysis of research projects submitted by ZOU undergraduate 
students between 2000-2009. The study aimed to achieve two objectives. The first 
objective was to conduct a conceptual analysis of the projects in order to establish the 
status of students' research work, its gaps and areas of saturation. The present study 
found out that for the past decade, undergraduate research work at the ZOU has been 
characterized by four features. 

> A clear distinction existed, between research methodologies applied by 
students in the faculties of Arts & Education, Commerce & Law and Social 
Sciences on one hand and those applied by students in the faculty of Physical 
Sciences on the other hand. And in the faculty of Physical Sciences another 
distinction existed between the departments of Agriculture and Geography. 
Whilst (%) of studies in the department of Agriculture were quantitative and 
used simple experiments, (%) of studies in the department of Geography were 
case studies and employed mixed methods. 

> The study also found out that although most research studies from the faculties 
of Arts and Education, Commerce and Law and Social Sciences were qualitative, 
basic descriptive surveys, they also employed quantitative techniques in data 
collection, analysis and presentation. 

> A significant number (22%) of projects across faculties did not identify the 
research paradigm. 

> Most research projects in the faculties of arts and education. Social Sciences, 
Commerce and Law were guided by descriptive research questions, whereas 
projects in the faculty of Physical Sciences were guided by relational research 
questions and hypotheses. 

> Questionnaires and interviews were the most preferred data collection 
instruments for studies in the faculties of Arts and Education, Commerce and 
Law and Social Sciences. Studies in the faculty of Physical Sciences used tests 
and observations. 

The second objective was to conduct a relational analysis of the projects in order to assess 
the methodological appropriateness and methodological quality of the projects. The 
present study revealed that students' projects demonstrated a high degree of 
methodological appropriateness and methodological quality. Appropriate linkages were 
established between research questions, data collected and the methods used in data 
collection, analysis and presentation. The use of mixed methods also enhanced the quality 
and appropriateness of the research methodology. 

However, the confinement of students' projects to basic descriptive surveys (for 
qualitative studies) and to simple experiments (for quantitative studies) reflects limited 
knowledge and low- level research skills in both research students and their supervisors. 
Thus the present study raises a number of critical questions: What is the level and quality 
of knowledge and research skills ZOU students have? What is the quality of the 
preparatory courses ZOU students undertake before embarking on their research projects? 
Do these courses adequately equip students with the knowledge and research skills 
students require for them to undertake and produce quality research projects? What is the 
quality of supervision research ZOU students get during the conduct of their research 
projects? How can ZOU enhance the quality of project supervision, quality of students' 
research work and quality assessment of students through research projects? These 
questions call for further research studies. 
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Implications 

The current study has a number of implications in the assessment of students through 
research projects. Primarily, it serves to inform the Zimbabwe Open University about the 
status of students' research work for the past ten years. This study contributes knowledge 
on how the current developments taking place in research theory have influenced research 
activities at the ZOU. 

The study also points in the direction educational students' research activities at the ZOU 
are taking. The study contributes useful knowledge and skills that can be used in the 
assessment of ODL students through research work. Research students and their 
supervisors can use the relational analysis model developed in this study to assess quality 
of their own research work. 


Developers of teaching and learning materials can also use findings of this study to 
develop courses and materials that adequately equip students with the right knowledge 
and skills required for research. 

There is, at the ZOU, an urgent need to evaluate the current courses that prepare students 
for research work. These courses are: 


> Introduction To Research Methods 

> Statistics For Educational Research 

> Computer Application In Educational Research 
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